Study of Spectro-temporal Parameters in Musical Performance for Expressive Instrument Synthesis

Modeling expressive synthesis requires to deal with a wide range of sound’s behavior that include spectral dynamics which are idiomatic to the instrument or characteristic of the playing technique. In the paper we analyze short time evolution of power-spectral envelopes, adaptively changing the analysis parameters so as to variably include filter envelopes, and the pitched excitation parts. Typical sequences of sound features are obtained from quantized sequences of parameters. By applying string matching techniques, new sequences of features are created and new sounds that have dynamically changing temporal behavior are re-synthesized. 1. I N T R O D U C T I O N Simulation of naturally behaving sounds, such as sounds t.hat occur during an expressive musical performance. requires modeling and understanding of the spectro-temporal behavior in a variety of playing conditions. In a natural sound we observe a great amount of spectro-temporal variations, which are difficult to capture and model. Thus, achieving an appropriate synthesis is not an easy task. Most of studies in instrumental sound deal with timbre of isolated sounds recorded in ”laboratory” conditions, thus having practically fixed pitch and limited dynamics. The problem of expressive synthesis requires to deal with a much wider range of sound’s behavior, so called expressive inflections. One must understand the nature of sound’s behavior in a variety of playing conditions and be able l,o quantify the changes of spectro-temporal parameters in relation to the expressive events. These new questions are idiomatic to the specific instrument (involving subtle sound properties such as noise or ”texture” of a sound), and may-be even related to the style of playing by a specific performer. As such, these questions usually are not considered to be in the ’*timbre” domain. One of the problems in modeling sound is the multitude of features that are relevant for our perception. These include sound intensity, overall spectral envelope that defines the color of the sound and longer correlations that correspond to pitch / repetitive properties in the signal. A convenient representation of the different spectral features is by means of cepstral c0efficient.s [2],[3]. For a power-spectrum S(w) the cepstral coefficients represent the Xavier Rodet Anal ysis/S y n t hesis Team IRCAM, Paris 75004 France Fourier series expansion of ZogS(w)